- Posted on
- Featured Image
Step-by-step guide to running private, on-prem AI on Rocky Linux: why its RHEL compatibility, SELinux, and repos (EPEL/CRB) make it ideal; prep via dnf/apt/zypper; optional NVIDIA/ROCm; Python venv with PyTorch/Transformers/ONNX; hands-on sentiment script, llama.cpp local LLM, and FastAPI REST service with systemd; optional Ollama; performance, security, containers, and reproducibility tips; a repeatable workflow to production.